Papers with bag-of-words models
Recovering Lexically and Semantically Reused Texts (2021.starsem-1)
Copied to clipboard
| Challenge: | Writers often repurpose material from existing texts when composing new documents. |
| Approach: | They propose to use local text reuse detection to detect localized regions of lexically or semantically similar text embedded in otherwise unrelated material. |
| Outcome: | The proposed methods perform better on three LTRD tasks, detecting plagiarism, modeling journalists’ use of press releases, and identifying scientists’ citation of earlier papers. |
VICTOR: a Dataset for Brazilian Legal Documents Classification (2020.lrec-1)
Copied to clipboard
Pedro Henrique Luz de Araujo, Teófilo Emídio de Campos, Fabricio Ataides Braz, Nilton Correia da Silva
| Challenge: | Approximately 10% of these are unstructured and requiring a lot of time to sort through. |
| Approach: | They propose to use a dataset built from Brazil's Supreme Court digitalized legal documents to improve document type classification and theme assignment tasks. |
| Outcome: | The proposed dataset is based on 45 thousand appeals and contains roughly 692 thousand documents—about 4.6 million pages. |
Convolutional Neural Network for Universal Sentence Embeddings (C18-1)
Copied to clipboard
| Challenge: | Recent studies show that averaging word embeddings is effective for NLP but these models represent a sentence only in terms of features of words or uni-grams. |
| Approach: | They propose a CNN-based model that uses both features of words and n-grams to encode sentences. |
| Outcome: | The proposed model performs better than existing models in transfer learning setting and exceeds state of the art in supervised learning setting by initializing the parameters with the pre-trained sentence embeddings. |
Pretrained Transformers Improve Out-of-Distribution Robustness (2020.acl-main)
Copied to clipboard
| Challenge: | Pretrained Transformers are more effective at detecting anomalous or OOD examples, while many previous models are frequently worse than chance. |
| Approach: | They construct a new robustness benchmark with real distribution shifts to measure out-of-distribution generalization for seven NLP datasets and compare them to previous models. |
| Outcome: | The proposed model generalizations for seven datasets show that pretrained Transformers are significantly less effective at detecting anomalous or OOD examples, while many previous models are often worse than chance. |
The Effects of Corpus Choice and Morphosyntax on Multilingual Space Induction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on inductive biases of language models towards natural language has focused on quantifying their ability to build multilingual spaces. |
| Approach: | They propose to use linguistically motivated tasks as a proxy to study inductive biases of language models with respect to natural language phenomena to build multilingual embedding spaces. |
| Outcome: | The proposed model performance is compared with other models using a set of linguistically motivated tasks and a training corpus in 15 languages. |
Decoding a Neural Retriever’s Latent Space for Query Suggestion (2022.emnlp-main)
Copied to clipboard
Leonard Adolphs, Michelle Chen Huebscher, Christian Buck, Sertan Girgin, Olivier Bachem, Massimiliano Ciaramita, Thomas Hofmann
| Challenge: | Neural retrieval models have replaced bag-of-words methods for document retrieval . however, they lack the interpretability of bag-off-word models . |
| Approach: | They train a query decoder that generates a meaningful query from a latent representation of a neural search engine. |
| Outcome: | The proposed model outperforms both query reformulation and PRF information retrieval baselines. |
Improving Answer Selection and Answer Triggering using Hard Negatives (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to answer selection and answer triggering have been proposed. |
| Approach: | They propose to use hard negatives with a siamese network and a suitable loss function for answer selection and answer triggering. |
| Outcome: | The proposed model improves on InsuranceQA, SelQA, and an internal QA dataset by 2.3 points over previous baselines. |
STAIR: Learning Sparse Text and Image Representation in Grounded Tokens (2023.emnlp-main)
Copied to clipboard
Chen Chen, Bowen Zhang, Liangliang Cao, Jiguang Shen, Tom Gunter, Albin Jose, Alexander Toshev, Yantao Zheng, Jonathon Shlens, Ruoming Pang, Yinfei Yang
| Challenge: | State-of-the-art contrastive learning models like CLIP and ALIGN are less interpretable and suffer from inferior accuracy than dense representations. |
| Approach: | They extend CLIP and ALIGN models to build a sparse semantic representation that is interpretable and easy to integrate with existing retrieval systems. |
| Outcome: | The proposed model outperforms CLIP and ALIGN models on image and text retrieval tasks with a 4.9% and +4.3% improvement on COCO-5k textimage and imagetext retrieval respectively. |